Method and apparatus for sectioning image into plurality of regions

ABSTRACT

A method and apparatus for sectioning an image into a plurality of regions, by which regions of an image having various features can be extracted stably in a form similar to what humans can perceive, are provided. The method for sectioning an input image into a plurality of regions includes the steps of: converting each pixel of the input image into color coordinates in an arbitrary color space including a luminance (L) axis; forming a plurality of layers and a plurality of bins from the color space; if a distance between the centers of upper last and lower last circles determined using a color, a texture, and a position on an image plane for a pixel is less than a first threshold value, including the upper last and lower last circles in the same cluster and performing the above step on an adjacent lower layer; if the distance between the centers of the upper last and lower last circles is no less than the first threshold value or if the adjacent lower layer does not exist, partitioning the cluster using a last circle including the smallest number of pixels in the color space used in calculating an average color, an average texture, and an average position among the last circles included in the cluster; and performing the above steps on the remaining pixels not included in the upper last or the lower last circle and generating an image graph using clusters obtained for all pixels in the color space.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention relates to image processing, and moreparticularly, to a method and apparatus for sectioning an image into aplurality of regions.

[0003] 2. Description of the Related Art

[0004] Important factors in forming an image include color,illumination, shape, texture, positions of the objects, mutual geometryof the objects, and the position of the observer relative to an imageforming apparatus. Image formation is affected by various devices andenvironmental conditions. Ideal image segmentation involves effectivelydistinguishing a meaningful object or a region of the same color fromother objects or background in a form similar to what humans canrecognize, regardless of the above conditions for image formation. Anumber of conventional image segmentation techniques for accomplishingthis have been suggested.

[0005] One of the conventional image segmentation techniques isdisclosed in “Distribution Free Decomposition of Multivariate data” [D.Comaniciu and P. Meer, Pattern Analysis and Applications, 1999, Vol, 2,pp 22-30]. The conventional image segmentation technique disclosedtherein includes the use of a point-clustering algorithm. Thepoint-clustering algorithm is applied to detect a distribution modeamong distributions of color points in a color space, that is, groups ofpixels having a color similar to that of a target pixel in an imageusing color information about the target pixel and obtain a region ofthe same color by projecting all pixels spaced apart within apredetermined distance onto the image. While this conventional imagesegmentation technique can quickly distinguish a region of the samecolor, it has a problem in that an image is excessively segmented in aregion containing a texture. Furthermore, another problem encountered bythe conventional image segmentation is that it is difficult to properlydistinguish a region of a three-dimensional object having shades byinsufficiently considering a shading effect

[0006] Another conventional image segmentation approach for consideringa shading effect is disclosed in U.S. Pat. No. 5,933,524 entitled“Method for Segmentation of Digital Color Images”. This conventionalapproach includes the use of line-clustering algorithm, in which colorhistograms are used for segmentation of a color image. The conventionalimage segmentation disclosed in the above cited reference involvesobtaining a color histogram for a specific object and converting thecolor histogram into a normalized color histogram. The reason is toreduce changes in a color histogram of an object that may occur in alight source under other conditions. Then, thus-obtained color clustersin the color space are described as a parametric function by threemodels. Here, a parametric function that defines a specific object isused for a measurement of distances between locations of color pixels inthe color space thereby effectively distinguishing the specific objectin the image. However, this conventional image segmentation approach haslimitations in segmenting regions of actual image having various colordistribution. Furthermore, the conventional image segmentation hasproblems in that complicated computations are performed for calculatingeigenvalues required for producing a model which is most suitable forrepresenting distributions in a three-dimensional space, and therefore atime taken for segmentation of color image becomes longer.

SUMMARY OF THE INVENTION

[0007] To solve the above problems, it is a first objective of thepresent invention to provide a method of sectioning an image into aplurality of regions, by which image regions having various features canbe extracted accurately in a form similar to what humans can perceive.

[0008] It is a second objective of the present invention to provide anapparatus for sectioning an image into a plurality of regions thatperforms the image sectioning method.

[0009] In order to achieve the first objective, the present inventionprovides a method for sectioning an input image into a plurality ofregions. The method includes the steps of: (a) converting each pixel ofthe input image into color coordinates in an arbitrary color spaceincluding a luminance (L) axis; (b) partitioning the color space into aplurality of layers along the luminance (L) axis and partitioning eachof the plurality of layers into a plurality of bins along color (α andβ) axes; (c) obtaining an upper last circle by a mean-shift analysis,the upper last circle encircling the largest number of pixels havingfeatures most similar to those of a base pixel located at apredetermined position within a bin having the largest number of pixelsin the color space among bins included in an upper layer; (d) obtaininga lower last circle by the mean-shift analysis, the lower last circleincluding the largest number of pixels having features most similar tothose of a base pixel located at the same position as the center of theupper last circle in a lower layer adjacent to the upper layer; (e)including the upper last and lower last circles in the same cluster if adistance between the centers of the upper last and lower last circles isless than a first threshold value, and if another lower layer adjacentto the lower layer exists, determining the lower layer and the adjacentlower layer as a new upper layer and a new lower layer, respectively,and returning to step (d); (f) if the distance between the centers ofthe upper last and lower last circles is no less than the firstthreshold value or if the adjacent lower layer does not exist,partitioning the cluster using a last circle including the smallestnumber of pixels in the color space used in calculating an averagecolor, an average texture, and an average position among the lastcircles included in the cluster; and (g) performing steps (c)-(f) on theremaining pixels not included in the upper last or the lower last circleand generating an image graph using clusters obtained for all pixels inthe color space.

[0010] In order to achieve the second objective, the present inventionprovides an apparatus for sectioning an input image into a plurality ofregions. The apparatus includes: an image preprocessor that smooths theinput image a predetermined number of times, enhances edges in thesmoothed image, and outputs a preprocessed image; a feature valuecalculator that calculates color feature values and texture featurevalues from the image output from the image preprocessor on apixel-by-pixel basis and outputs the calculated result; a main regionsection unit that generates an image graph from clusters obtained byusing color, texture, and position on an image plane for each pixel inthe image output from the image preprocessor and the color and texturefeature values output from the feature value calculator and outputs thegenerated image graph; and an image graph simplification unit thatcompares a color-texture distance calculated using the color featurevalues and the texture feature values output from the feature valuecalculator with a second threshold value, simplifies the image graphaccording to the comparison result, and outputs a final image graphobtained from the simplified image graph.

BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above objectives and advantages of the present invention willbecome more apparent by describing in detail preferred embodimentsthereof with reference to the attached drawings in which:

[0012]FIG. 1 is a flowchart for explaining a method for sectioning animage into a plurality of regions according to the present invention;

[0013]FIG. 2 is block diagram of an apparatus for sectioning an imageinto a plurality of regions according to a preferred embodiment of thepresent invention;

[0014]FIG. 3 shows an Lαβ color space for explaining layers and bins;

[0015]FIG. 4 is a flowchart for explaining a mean-shift analysisaccording to the present invention, by which the step 14 shown in FIG. 1is performed;

[0016] FIGS. 5(a) and (b) shows a mean-shift analysis;

[0017]FIG. 6 is a flowchart for explaining a mean-shift analysisaccording to the present invention, by which the step 16 shown in FIG. 1is performed;

[0018]FIG. 7A shows an example of a cluster formed by the method andapparatus for sectioning an image into a plurality of regions accordingto the present invention and FIG. 7B is a graph for explaining a localminimum;

[0019]FIG. 8 is a exemplary diagram for explaining a fundamental regiondivision map;

[0020]FIG. 9 is an exemplary diagram of an image graph generated fromthe fundamental region division map of FIG. 8;

[0021]FIG. 10 is a block diagram of an apparatus for measuring acolor-texture distance used in the step 30 shown in FIG. 1;

[0022]FIG. 11 is a block diagram of the color-texture distance generatorof FIG. 10;

[0023]FIG. 12 is a block diagram of an image graph simplification unit;and

[0024]FIGS. 13A, 14A and 15A show input or preprocessed images, and

[0025]FIGS. 13B, 14B, and 15B show regions into which an image issectioned by the method and apparatus for sectioning an image into aplurality of regions according to the present invention.

DETAILED DESCRIPTION OF THE INVENTION

[0026] Referring to FIG. 1, a method for sectioning an image into aplurality of regions includes converting each pixel of an input imageinto color coordinates in an Lαβ color space and partitioning the colorspace into layers and bins (steps 10 and 12), obtaining an upper lastcircle and a lower last circle using a mean-shift analysis (steps 14 and16), and forming and partitioning clusters (steps 18-26), and generatingand simplifying an image graph using clusters (steps 28-30).

[0027] Referring to FIG. 2, an apparatus for performing the method forsectioning an image into a plurality of regions shown in FIG. 1 includesan image preprocessor 32, a feature value calculator 34, a main regionsection unit 36, and an image graph simplification unit 38.

[0028] Referring to FIGS. 1 and 2, the main region section unit 36converts each pixel of the input image into color coordinates in the Lαβcolor space (step 10). In this case, L is luminance, and α and β arecolor components, all of which are defined by Equation (1):$\begin{matrix}{{L = \frac{r + g + b}{3}}{\alpha = {\left( {r - g} \right) \cdot 2}}{\beta = \left( {r + g - {2b}} \right)}} & (1)\end{matrix}$

[0029] where r, g and b denote color values of each pixel.

[0030] In step 10, the main region section unit 36 may input an imagethrough an input terminal IN1 or from the image preprocessor 32 as shownin FIG. 2. For example, the image preprocessor 32 repeatedly smooths theimage input through the input terminal IN1 a predetermined number oftimes and outputs an image preprocessed by enhancing edges in thesmoothed image to the feature value calculator 34 and to the main regionsection unit 36. As a consequence, the main region section unit 36 canconvert the smoothed image output from the image preprocessor 32 or eachpixel of the image input through the input terminal IN1 into colorcoordinates in the Lαβ color space.

[0031] Smoothing of an input image performed by the image preprocessor32 will now be described in detail.

[0032] In general, an input image such as that captured through a camerasensor or a video image signal is composed of two portions: noise and ameaningful signal. Here, each component of an image includes differentkinds of information. This method for statistically dividing twoportions allows the input image to be broken into various differentcomponents. For example, the image preprocessor 32 performs anedge-preserving image smoothing process in such a way as to filter outnoise by smoothing the input image and to preserve edges of an object bydecomposing the smoothed image into a portion having an outstanding edgeand a portion having no outstanding edge. This edge-preserving imagesmoothing is disclosed in “Two Tasks in Image Enhancement Technology” byP. A. Chochia, Optical Memory and Neural Networks, 1998, and summarizedas follows.

[0033] If a set {x} of pixels in a window of a predetermined size isdefined by Equation (2), windows of different sizes are applied tosmooth pixels excluding those having outstanding luminance values amongpixels in each window.

{x}=x ₁ ≦x ₂ , . . . , ≦x _(n)(x _(i) εW _(nm))  (2)

[0034] Luminance value I^(q) _(mn) obtained after smoothing of an inputimage is repeated q times is calculated in a window W^(q) _(mn) usingthe previously obtained luminance value I^(q−1) _(mn), as expressed byEquation (3): $\begin{matrix}{1_{mn}^{q} = \frac{\sum\limits_{x \in W_{mn}^{q}}{{xw}\left( {x - 1_{mn}^{q - 1}} \right)}}{\sum\limits_{x \in W_{mn}^{q}}{w\left( {x - 1_{mn}^{q - 1}} \right)}}} & (3)\end{matrix}$

[0035] Here, w(x′) is a predetermined weight function. If x′ is between−σ and +σw(x′)=1, and if not, w(x′)=0. The luminance value I^(q−1) _(mn)refers to a central value of the weight function w(x′) in a previouswindow W^(q−1) _(mn). x_(mn) is used in place of I^(q−1) _(mn) ininitial smoothing of the input image. In this case, a luminance values′_(mn) of a pixel which has been smoothed Q times is represented byI^(Q) _(mn). That is, the luminance value s′_(mn) refers to a result ofsmoothing a pixel with Q windows. It is experimentally shown thatsmoothing is preferably performed twice. A square window having sides oflength 3-5 pixels was used for first smoothing, and a square windowhaving sides of length 10-40 pixels was used for second smoothing. Theuse of these windows has proven effective in extracting smoothedcomponents from an image which contains a large amount of noise, and inthis case, an acceptable mean square error of a computed result valuewas within 0.5%.

[0036] Following step 10, the Lαβ color space is partitioned into layersalong an L-axis, and each layer is partitioned again into bins along α-and β-axes (step 12). In this case, the positions X and Y of pixelscontained in each layer on an image plane are stored.

[0037] The layers and bins formed in step 12 will now be described withreference to FIG. 3. Referring to FIG. 3, which shows the Lαβ colorspace for layers and bins, the Lαβ color space is partitioned into aplurality of layers 40, 42, 44, . . . , and 46, and each of theplurality of layers 40, 42, 44, . . . , and 46 is partitioned into aplurality of bins. For example, the layer 40 is composed of a pluralityof bins 50, 51, 52, 53, 54, . . . , the layer 42 is composed of aplurality of bins 60, 61, 62, 64, . . . , and the layer 46 is composedof a plurality of bins 70, 71, 72, 74, . . . Each axis has a range from0 to 2^(P′)−1, and each bin is a cube whose length, width, and heightare p″.

[0038] First, it is assumed that the L-axis defines 2 to the 8th power(256) different brightness (p′=8), the α- and β-axes define 2 to the 8thpower (256) different colors (p′=8), and p″=4. It is further definedthat, of two layers shown in FIG. 3, one layer having lower brightnessis a “lower layer” and the other layer having higher brightness is an“upper layer”. That is, the layer 40 is an upper layer to the layer 42,and the layer 42 is a lower layer to the layer 40. Similarly, the layer42 is an upper layer to the layer 44, and the layer 44 is a lower layerto the layer 42.

[0039] Under the above assumption, four brightness values on the L-axisin the color space with L, α, and β coordinate axes are assigned foreach layer, which partitions the color space into 64 layers 40, 42, 44,. . . , and 46. That is, brightness values 255, 254, 253, and 252 areassigned to the layer 40. Similarly, the color space is partitionedalong the α- and β-axes such that each of the layers 40, 42, 44, . . . ,and 46 has 64×64 bins.

[0040] Following step 12, an upper last circle, in which pixels havingfeatures most similar to that of an initial pixel located at apredetermined position within a local maxima among the bins 50, 51, 52,53, 54, . . . contained in the first upper layer 40 shown in FIG. 3 aremost densely arrayed, is obtained by mean-shift analysis (step 14).Here, the local maxima refers to a bin including the largest number ofpixels among bins included in a particular layer.

[0041] A preferred embodiment of a mean-shift analysis used when themain region section unit 36 performs step 14 will now be described withreference to FIGS. 4 and 5. Referring to FIG. 4, a mean-shift analysismethod according to the present invention includes calculating anaverage color, an average texture, and an average position X and Y on animage plane for pixels selected among pixels contained in an upperarbitrary circle, comparing the average color, the average texture andthe average position for the selected pixels with those for a basepixel, and determining whether the averages of the selected pixelsconverge (steps 90-96). Steps 90-96 are repeated until a converged upperarbitrary circle is obtained. The method further includes determiningthe converged upper arbitrary circle as an upper last circle (step 98).

[0042] Referring to FIGS. 5(a) and (b) which explain a mean-shiftanalysis method, an upper layer 120 in FIG. 5(a) consists of one or moreupper arbitrary circles 106, 108, . . . , and 110, while a lower layer130 in FIG. 5(b) consists of one or more lower arbitrary circles 132,134, . . . , and 136. Here, the abscissa of each layer 120 or 130 is a βaxis and the ordinate is an α axis, which are same as the α- and β-axesin the color space shown in FIG. 3. In order to explain the mean-shiftanalysis of the present invention, the layers 120 and 130 are shown aseach having a plurality of arbitrary circles 106, 108, 110, 132, 134,and 136. However, each layer 120 or 130 may have only a single arbitrarycircle.

[0043] First, among pixels included in the first upper arbitrary circle106 having a center at a predetermined location 100 in the upper layer120 having the highest brightness as shown in FIG. 5(a) and a radius ofσ_(C), only pixels whose texture feature values are less than a texturefeature value of a pixel located at the center 100 of the first upperarbitrary circle 106 (hereinafter called “base pixel”) by a texturedistance σ_(θ) and which are spaced apart less than a predetermineddistance σ_(G) from the position of the base pixel on an image plane,are selected (step 90). The texture feature values of each pixelrequired for performing step 90 are output from the feature valuecalculator 34 of FIG. 2.

[0044] The operation of the feature value calculator 34 of FIG. 2 willnow be described. The feature value calculator 34 computes color featurevalues and texture feature values on a pixel-by-pixel basis from thepreprocessed image output from the image preprocessor 32, and outputsthe computed color and texture feature values to the main region sectionunit 36 and the image graph simplification unit 38.

[0045] With reference to the color feature values computed in thefeature value calculator 34, first, a color feature space and a texturefeature space are used together to provide a basis for a general featurespace. In this case, color feature values of each pixel in an image canbe specified by defining the color and texture feature spaces. The colorfeature values, i.e., brightness (B), hue (H), and saturation (S) ofeach pixel, are defined by Equations (4): $\begin{matrix}\begin{matrix}{B = \quad \sqrt{\frac{r^{2} + g^{2} + b^{2}}{3}}} \\{{H = \quad {\frac{120{^\circ}\quad \left( {b - u} \right)}{g + b - {2u}} + {60{^\circ}}}}\quad,{{{if}\quad r} = u}} \\{\quad {{\frac{120{^\circ}\quad \left( {r - u} \right)}{b + r - {2u}} + {180{^\circ}}}\quad,{{{if}\quad g} = u}}} \\{\quad {{\frac{120{^\circ}\quad \left( {g - u} \right)}{r + g - {2u}} + {300{^\circ}}}\quad,{{{if}\quad b} = u}}} \\{S = \quad {1 - \frac{u}{r + g + b}}}\end{matrix} & (4)\end{matrix}$

[0046] where u=min(r, g, b)

[0047] Next, with reference to the texture feature values calculated inthe feature value calculator 34, each of texture feature values ofmultiple scales and multiple orientations used for forming a texturefeature space is obtained by calculating a local variation v and a localoscillation f for each pixel in multiple directions and combining them.This texture image analysis is called a Gray Level Difference Method.The Gray Level Difference Method is disclosed in “Some Basic TextureAnalysis Techniques” (E. J. Carton, J. S. Weszka and A. Rosenfeld,TR-288, Univ of Maryland, 1974) and “Texture Analysis Anno” (L. VanGool, P. Dewaele and A. Oosterlinck, CVGIP, 1985). The brightness (B) ofan image defined by Equation (4) is used in extracting the texturefeature values.

[0048] With regard to local variation v, pixels having length 2L′ havinga pixel (m, n) as a center exist in an image, and the pixels are turnedaround the pixel (m,n) at angle a′_(k)=kπ/K, m and n are the same as thespatial positions X and Y of a pixel on an image plane, and k=0, 1, . .. , K−1. Assuming that I_(i) represents the brightness (B) of a pixelamong pixels uniformly arrayed in this way, where −L′≦i≦L′, an upwardweight variation v⁺ and a downward weight variation v⁻ are expressed byEquations (5): $\begin{matrix}\begin{matrix}{{v^{+} = \quad {\sum\limits_{i = {- L}}^{L - 1}{w_{i}d_{i}}}},\quad {{{if}\quad {di}} > 0}} \\{{v^{-} = \quad {\sum\limits_{i = {- L}}^{L - 1}{w_{i}\left( {- d_{i}} \right)}}},\quad {{{if}\quad {di}} < 0}}\end{matrix} & (5)\end{matrix}$

[0049] where d_(i) denotes a gradient I_(i+1)−I_(i) between brightnessesof adjacent pixels in the pixel array and w_(i) is a cosine weightfunction Acos(iπ/(2L′+1). A coefficient A in the cosine weight functionw_(i) is used for establishing${\sum\limits_{i = {- L}}^{L}w_{i}} = 1.$

[0050] In this case, local variation v_(k) is designated as the smallerof the upward weight variation v⁺ and the downward weight variation v asexpressed by Equation (6):

v _(k)=min(v _(k) ⁺ , v _(k) ⁻)  (6)

[0051] A local oscillation f is defined by the number of d_(i) (d_(i)denotes the magnitude of oscillation), the magnitudes of which exceed apredetermined threshold value at the same time that the signs arechanged among d_(i) calculated along a length 2L′ of the array, where−L′≦i≦L′. In this case, local variation v_(k) and local oscillationf_(k) for each pixel are multiplied to obtain texture feature valuest_(k) ^(^) (=v_(k)f_(k)) for a corresponding pixel. Furthermore, to makethe calculated texture feature values more equal, Equation (7) is used:$\begin{matrix}{t_{k} = {\tan \quad {h\left\lbrack {\alpha {\sum\limits_{h}{{\hat{t}}_{k}(h)}}} \right\rbrack}}} & (7)\end{matrix}$

[0052] As is evident from Equation (7), the texture feature values t_(k)are smoothed as an average of a window having a size of h×1, and highand low texture feature values are made lower and higher, respectively,due to modification using a hyperbolic tangent function ‘tanh’. In thiscase, when the size of an image is reduced Z times (by a half everytime) at different frequencies, the texture feature values t_(k) foreach pixel defined by Equation (7) can be expressed by Equation (8):$\begin{matrix}{t_{k}^{z} = {\tan \quad {h\left\lbrack {\alpha {\sum\limits_{h}{{\hat{t}}_{k}^{z}(h)}}} \right\rbrack}}} & (8)\end{matrix}$

[0053] As is evident from Equation (8), KZ texture feature values t_(k)^(z) are produced for each pixel.

[0054] Turning to FIG. 4, after step 90, the main region section unit 36calculates an average color, an average texture, and an average positionon an image plane for the selected pixels (step 92). Here, the averagecolor means an average of color components α and β for the selectedpixels in the layer on the given L and is called average Lαβ. Theaverage texture means average of textures for the selected pixels, andthe average position means an average of positions for the selectedpixels. Then, when each of the calculated averages is compared with eachaverage of the base pixel 100 which is the center of the first upperarbitrary circle 106, it is determined whether a spacing degree is lessthan or equal to a first predetermined value (step 94). Here, in step94, the spacing degree is calculated from differences between theaverages obtained in step 92 and the averages of the base pixel 100 aswill be described below. If the calculated spacing degree is greaterthan the first predetermined value, a pixel, the center of which is aposition 102 corresponding to the average color obtained in step 92, isdesignated as an imaginary base pixel, and texture feature values and aposition on an image plane for the imaginary base pixel are designatedas the average texture and the average position calculated in step 92.Then, a new upper arbitrary circle, i.e., the second upper arbitrarycircle 108 having the center of the new base pixel 102 at the position102, is determined, and the process returns to step 90 (step 96).

[0055] Steps 90-96 are performed in the direction indicated by an arrow122 shown in FIG. 5(a) until the spacing degree is less than or equal tothe first predetermined value, that is, an average color, an averagetexture, and an average position on an image plane for selected pixelsin an upper arbitrary circle converge on feature values (color, texture,and position) of a base pixel of the upper arbitrary circle. That is, asshown in FIG. 5(a), if the spacing degree obtained from the differencesbetween the averages (color, texture, and position) for an upperarbitrary circle 110 and the features (color, texture and position) of abase pixel 104 at the center of the upper arbitrary circle 110, is lessthan or equal to the first predetermined value, the upper arbitrarycircle 110 is determined as an upper last circle 84 shown in FIG. 3(step 98). Here, the upper last circle 84 or 110 includes the largestnumber of pixels having feature values most similar to those of theimaginary base pixel 104.

[0056] Turning to FIG. 1, after step 14, an imaginary base pixel havingthe same features (texture and position) as the base pixel of the upperlast circle 84 is set at the same position α and β as the center of theupper last circle 84 in the lower layer 42 successively adjacent to theupper layer 40, and a lower last circle 82 including the largest numberof pixels having feature values most similar to those of the imaginarybase pixel is obtained by the mean-shift analysis (step 16).

[0057] A mean-shift analysis method according to a preferred embodimentof the present invention used when the main region section unit 36performs step 16 will now be described with reference to FIG. 6.

[0058] Referring to FIG. 6, the mean-shift analysis method according tothe present invention used in performing step 16 shown in FIG. 1includes obtaining an average color, an average texture, and an averageposition X and Y on an image plane for pixels, which are selected amongpixels contained in a lower arbitrary circle until a converged lowerarbitrary circle is obtained (steps 150-156), and determining theconverged lower arbitrary circle as a lower last circle (step 158).

[0059] Referring to FIGS. 5(a) and (b), among pixels included in thefirst lower arbitrary circle 132, having its center at the same location140 in the lower layer 130 as the center 104 of the upper last circle110 and having a diameter of σ_(c), only pixels whose texture featurevalues are less than those of the set base pixel 140 by σ_(θ) and whichare spaced apart less than a predetermined distance σ_(G) from theposition of the base pixel 140 on an image plane, are selected (step150). The texture feature values used in performing step 150 arecalculated by the feature value calculator 34 as described above.

[0060] In a preferred embodiment of the present invention, pixels areselected among pixels in an arbitrary circle in steps 90 and 150 in thefollowing manner. First, using differences ΔX and ΔY between positionson an image plane, differences Δα and Δβ between color components, anddifferences Δθ between texture feature values of any pixel located in anupper (or lower) arbitrary circle and a central pixel located at thecenter of that upper (or lower) arbitrary circle, a distance Δ′ betweenany pixel and the central pixel is obtained according to Equation (9):$\begin{matrix}{\Delta^{\prime} = {\frac{{\Delta \quad X^{2}} + {\Delta \quad Y^{2}}}{\sigma_{G}^{2}} + \frac{{\Delta\alpha}^{2} + {\Delta\beta}^{2}}{\sigma_{C}^{2}} + {\sum\limits_{i^{\prime} = 1}^{K}\frac{\sum\limits_{j^{\prime} = 1}^{Z}{\Delta\theta}_{i^{\prime}j^{\prime}}^{2}}{\sigma_{\theta_{i^{\prime}}}^{2}}}}} & (9)\end{matrix}$

[0061] where θ denotes an arrangement θ[KZ] of KZ texture feature valuesfor each pixel obtained from Equation (8) as a texture response, andσ_(θ) ₁ denotes a texture distance at an i-th size among K differentsizes.

[0062] If the distance Δ′ obtained as defined by Equation (9) is lessthan a second predetermined value, for example, ‘1’, the pixel isselected as a pixel considered for any average value. If not, the pixelis not selected.

[0063] After step 150, an average color, i.e., an average Lαβ, anaverage texture, and an average position on an image plane arecalculated for pixels selected among pixels in the first lower arbitrarycircle 132 (step 152). Here, each average is calculated in the same wayas described for step 92. After step 152, when each of the calculatedaverages is compared with each feature value of the base pixel at thecenter 140 of the first lower arbitrary circle 132, it is determinedwhether or not the spacing degree is greater than a first predeterminedvalue (step 154). Here, the spacing degree used for step 94 or 154 iscalculated by using Equation (9). That is, in step 94 or 154, in orderto obtain the spacing degree Δ′, differences between average positionobtained in step 92 or 152 for any upper (or lower) arbitrary circle andposition on an image plane of a central pixel in that upper (or lower)arbitrary circle are substituted for ΔX and ΔY, differences betweenaverage colors and color components of the central pixel are substitutedfor Δα and Δβ, and the difference between average texture and a texturefeature value of the central pixel is substituted for Δθ in Equation(9). If the calculated spacing degree is greater than the firstpredetermined value, a pixel at a position 142 corresponding to theaverage color obtained in step 152 is designated as an imaginary basepixel, and a texture feature value and a position on an image plane forthe base pixel are designated as the average texture and the averageposition calculated in step 152. Then, a new lower arbitrary circle,i.e., the second lower arbitrary circle 134 having the new base pixel atits center 142, is determined, and the process returns to step 150 (step156).

[0064] Steps 150-156 are performed in the direction indicated by anarrow 138 shown in FIG. 5(b) until the spacing degree is less than orequal to the first predetermined value, that is, an average color, anaverage texture, and an average position on an image plane for selectedpixels in a lower arbitrary circle converge on feature values (color,texture, and position) of a base pixel of the upper arbitrary circle.That is, as shown in FIG. 5(b), if the spacing degree obtained from thedifferences between the averages (color, texture, and position) for alower arbitrary circle 136 and the features (color, texture andposition) of a base pixel at the center 144 of the lower arbitrarycircle 136 is less than or equal to the first predetermined value, thelower arbitrary circle 136 is determined as a lower last circle 82 shownin FIG. 3 (step 158). Here, the upper last circle 82 or 136 includes thelargest number of pixels having feature values most similar to those ofthe base pixel 144.

[0065] As described above, since a layer having lower brightness and alayer having higher brightness are defined as a ‘lower layer’ and an‘upper layer’, respectively, the last circle for the layer having lowerbrightness is obtained after the last circle for the layer having higherbrightness is obtained in steps 14 and 16. Conversely, if the layerhaving higher brightness and the layer having lower brightness aredefined as a “lower layer” and an “upper layer”, respectively, a lastcircle for the layer having higher brightness is obtained after a lastcircle for the layer having lower brightness is obtained in steps 14 and16.

[0066] Turning to FIG. 1, after step 16, an upper last circle determinedin an upper layer is selectively connected to a lower last circledetermined in a lower layer (step 18) to cluster pixels having similarsaturations and gradually varying brightnesses due to a shading effect(step 20). For example, after step 16, it is determined whether adistance [(α₂−α₁)²+(β₂−β₁)²]^(½) between the center (α₁, β₁) 104 of theupper last circle 110 and the center (α₂, β₂) of the lower last circle136 is less than a first threshold value th1 (step 18). If the distanceis less than the first threshold value th1, the upper last and lowerlast circles 110 and 136 (or 84 and 82) are included in the same cluster86 (step 20).

[0067]FIG. 7A shows an example of a cluster 160 formed by the method andapparatus for sectioning an image into a plurality of regions shown inFIGS. 1 and 2, in which the cluster 160 includes a plurality of lastcircles 170, 172, 174, 176, 178, and 180. FIG. 7B is a graph forexplaining a local minimum, in which the number of pixels is plottedagainst the layer.

[0068] After step 20, it is determined whether another lower layersuccessively adjacent to the lower layer exists (step 22). That is, itis determined whether a last circle is obtained for the last layer 46.If an adjacent lower layer exists, that is, unless a last circle isobtained for the last layer 46, the process returns to step 16. In thiscase, steps 16-22 are performed on the lower layer, which is a new upperlayer, and the adjacent lower layer, which is a new lower layer. Forexample, in a case where steps 16-22 are performed for obtaining a lastcircle for another lower layer 44, the previous lower layer 42 is a newupper layer and the lower layer 44 is a new lower layer.

[0069] However, if the distance between the center (α₁, β₁) 104 of theupper last circle 110 and the center (α₂, β₂) of the lower last circle136 is no less than the first threshold value th1, or if an adjacentlower layer does not exist, the cluster 160 is partitioned using a lastcircle including the smallest number of pixels in a color space used incalculating an Lαβ average among the last circles 170, 172, 174, 176,178, and 180 included in the cluster 160 shown in FIG. 7A (step 24).That is, if the distance is no less than the first threshold value th1,there is no adjacent lower layer or one cluster ceases to be formed andthen step 24 is performed. Step 24 is performed to minimize an errorcaused by including various objects having similar chromaticcharacteristics but different brightnesses in the same cluster. Forexample, if the last circles 170, 172, 174, 176, 178, and 180 use 5, 15,23, 17, 34, and 14 pixels, respectively, in order to calculate the Lαβaverage, the cluster 160 shown in FIG. 7A is partitioned using the lastcircle 176 including only 17 pixels. Here, the circle 176 corresponds toa local minimum mode 190 as shown in FIG. 7B.

[0070] After step 24, it is determined whether pixels not included inany last circle 84, 82 . . . , or 80 exist in any layer 40, 42, 44, . .. , and 46 (step 26). If pixels not included in any last circle 84, 82 .. . , or 80 exist in each of the layers 40, 42, 44, . . . , and 46, theprocess returns to step 14 in order to perform steps 14-24 on theremaining pixels not included in any last circle 84, 82 . . . , or 80.However, if such pixels do not exist, an image graph is generated from afundamental region division map obtained by using clusters available forall pixels in the color space (step 28).

[0071]FIG. 8 is an example of a fundamental region division mapconsisting of five regions 200, 202, 204, 206, and 208 which areassigned region numbers 1, 2, 3, 4, and 5, respectively. FIG. 9 is anexample of an image graph generated from the fundamental region divisionmap shown in FIG. 8. As is evident from FIG. 9, adjacent regions on thefundamental region division map are connected to one another. Here,{circle over (1)}, {circle over (2)}, {circle over (3)}, {circle over(4)}, and {circle over (5)} denote the five regions 200, 202, 204, 206,and 208 shown in FIG. 8, respectively.

[0072] The main region section unit 36 generates the fundamental regiondivision map shown in FIG. 8 from the clusters obtained by performingsteps 10-26, and generates the image graph of FIG. 9 using thefundamental region division map (step 28). In this case, the main regionsection unit 36 stores information about each region such as the numberof pixels, an average color, and texture feature values. In particular,the average color is obtained averaging brightnesses, hues, andsaturations (BHS) of pixels calculated by the feature value calculator34. Here, the image graph of FIG. 9 provides information about whatregions are adjacent to each other in the fundamental region divisionmap of FIG. 8 and information about weight values wt1, wt2, wt3, wt4,and wt5 for lengths of edges between adjacent regions.

[0073] After step 28, the image graph simplification unit 38 of FIG. 2compares the color-texture distance input through the input terminal IN2with a second threshold value th2, merges regions marked in the imagegraph output from the main region section unit 36 according to thecomparison result to generate a final image graph, and outputs thegenerated final image graph through the output terminal OUT1 (step 30).Step 30 will now be described in detail.

[0074]FIG. 10 is a block diagram of an apparatus for measuring acolor-texture distance used in performing step 30 shown in FIG. 1.Referring to FIG. 10, the apparatus includes a color distance calculator210, a texture distance calculator 212, and a color-texture distancegenerator 214. More specifically, the color distance calculator 210inputs color feature values of each of two points x and y such asbrightness (B), saturation (S), and hue (H) through an input terminalIN3 in a color feature space composed of color feature values of pixelsin an image. Then, the color distance calculator 210 assigns differentdegrees of importance to a difference between input brightnesses, adifference between input saturations, and a difference between inputhues, adds the brightness difference, the saturation difference, and thehue difference in proportion with the assigned degrees of importance,and outputs the added result to the color-texture distance generator 214as a color distance between the two points x and y. In this case, theabove degree of importance is determined in a color feature spacecomposed of BHS using the following three standard conditions. First,hue (H) is a more important factor in determining importance thanbrightness (B) and saturation (S) Second, if brightness (B) approachesan absolute value ‘0’, the color of an image is black without regard tohues (H) or saturation (S). Third, if saturation (S) approaches anabsolute value ‘0’, that is, the color of an image is gray, hue (H) hasan arbitrary value.

[0075] The color distance calculator 210 assigns degrees of importancedetermined on the basis of the above-mentioned standard conditions tothe brightness difference, the saturation difference, and the huedifference to obtain a color difference [D_(BHS1)(x, y)], expressed byEquation (10):

D _(BHS1)(x, y)=W _(B) |B(x)−B(y)|+F _(H)[min(S(x),S(y))](a{overscore(B)}+b){overscore (S)}|H(x)−H(y)|+F _(S)({overscore (S)}){overscore(B)}|S(x)−S(y)|  (10)

[0076] Here, B(o), H(o), and S(o) denote color feature values, i.e.,brightness, hue, saturation of point o (o refers to x or y),respectively, and {overscore (B)} and {overscore (S)} denote the averageof B(x) and B(y) and the average of S(x) and S(y), respectively. W_(B),a and b are constants, and F_(S)(j)) and F_(H)(j) denote linearcorrection functions for saturation and hue, respectively. In this case,the linear correction function F_(S)(j) or F_(H)(j) is employed tosuppress the difference between hue and saturation under conditions oflow brightness and low saturation. For this purpose, the linearcorrection function F_(S)(j) or F_(H)(j) is j if j is less than 1, and 1if j is 1 or greater.

[0077] As is evident from Equation (10), the brightness difference[B(x)−B(y)], the hue difference [H(x)−H(y)], and the saturationdifference [S(x)−S(y)] have different coefficients which are determinedaccording to the degrees of importance as described above.

[0078] In this case, the color distance calculator 210 may obtain acolor distance [D_(BHS2) ²(x, y)] by assigning degrees of importancedetermined based on the above-mentioned standard conditions to thebrightness difference, the saturation difference, and the hue differenceexpressed by Equation (11) in place of Equation (10):

D _(BHS2) ²(x, y)=W _(B) [B(x)−B(y)]² +W _(H) F _(B) [B(x),B(y)]F _(S)[S(x),S(y)][H(x)−H(y)]² +W _(S) F _(B) [B(x),B(y)][S(x)−S(y)]²  (11)

[0079] where W_(H) and W_(S) are constants, and F_(S) (•, •) and F_(B)(•, •) denote nonlinear correction functions for saturation andbrightness, respectively.

[0080] As is evident from Equation (11), the square of brightnessdifference [B(x)−B(y)]², the square of hue difference [H(x)−H(y)]², andthe square of saturation difference [S(x)−S(y)]² have differentcoefficients which are determined according to degrees of importance asdescribed above.

[0081] If the color distance calculator 210 calculates a color distanceusing Equation (10) or (11), image segmentation can be stably made in aregular way without being affected by the change in brightness.Furthermore, in order to enhance discrimination of low brightnessregion, the color distance calculator 210 may compute a color distance[D_(BHS3)(x, y)] as defined by Equation (12): $\begin{matrix}{{D_{BHS3}\left( {x,y} \right)} = {D_{BHS}*\left\{ {1.0 + \frac{\alpha^{\prime}}{{{\min \left( {{B(x)},{B(y)}} \right)}/256} + \beta^{\prime}} - \frac{\alpha}{1.0 + \beta^{\prime}}} \right.}} & (12)\end{matrix}$

[0082] where D_(BHS) denotes the color distance [D_(BHS1)(x, y)] or[D_(BHS2) ²(x, y)] expressed by Equation (10) or (11).

[0083] Meanwhile, the texture distance calculator 212 shown in FIG. 10inputs from the feature value calculator 34 texture feature values oftwo points x and y through an input terminal IN4 in a texture featurespace composed of texture feature values for pixels. Then, the texturedistance calculator 212 detects a difference between the input texturefeature values, calculates a texture distance between the two points xand y using weighting coefficients applied to multiple scales of atexture and the detected difference, and outputs the calculated texturedistance to the color-texture distance generator 214. For example, thetexture distance calculator 212 may calculate a texture distanceD_(t)(x, y) as expressed by Equation (13): $\begin{matrix}{{D_{t}\left( {x,y} \right)} = \left. {\sum\limits_{z = 1}^{Z}{W^{z}\sum\limits_{k = 1}^{K}}} \middle| {{t_{k}^{z}(x)} - {t_{k}^{z}(y)}} \right|} & (13)\end{matrix}$

[0084] where W^(z) denotes the weighting coefficient applied to themultiple scales of a texture.

[0085] The color-texture distance generator 214 computes a color-texturedistance using a color weight value w_(c) (u′, v′) and a texture weightvalue w_(t) (u′, v′) that vary depending on characteristics of segmentedregions u′ and v′. Here, the characteristics of the segmented regions u′an v′ mean a texture degree t(u′, v′), a size p(u′, v′) and a saturations(u′, v′) for the segmented regions u′ and v′ marked in the fundamentalregion division map of FIG. 8. That is, the color-texture distancegenerator 214 multiplies a variable color weight value w_(c) (u′, v′)and a variable texture weight value w_(t) (u′, v′) by the color distanceas expressed by Equation (10), (11), or (12) and the texture distance asexpressed by Equation (13), respectively, adds the multiplicationresults to each other, and outputs the added result to the image graphsimplification unit 38 as a color-texture distance through an outputterminal OUT2.

[0086]FIG. 11 is a block diagram of the color-texture distance generator214 shown in FIG. 10. Referring to FIG. 11, the color-texture distancegenerator 214 includes a color weight value calculator 220, a textureweight value calculator 222, first and second multipliers 224 and 226,and an adder 228.

[0087] The color weight value calculator 220 shown in FIG. 11 calculatesa color weight value w_(c) (u′, v′) as expressed by Equation (14) andoutputs the calculated color weight value to the first multiplier 224:

w _(c)(u′, v′)=ŵ _(c) +ŵ _(t)[1−t(u′, v′)p(u′, v′)]s(u′, v′)  (14)

[0088] where Ŵ_(c) and Ŵ_(t) denote color and texture weightingconstants, respectively. A texture degree t(u′, v′) is defined byEquation (15): $\begin{matrix}{{t\left( {u^{\prime},v^{\prime}} \right)} = \frac{{T\left( u^{\prime} \right)} + {T\left( v^{\prime} \right)}}{2*T_{\max}}} & (15)\end{matrix}$

[0089] where T_(max) denotes a maximum of texture degree. T(u′) isdefined by Equation (16): $\begin{matrix}{{T\left( u^{\prime} \right)} = {\sum\limits_{z = 1}^{Z}{W^{z}{\sum\limits_{k = 1}^{K}{t_{k}^{z}\left( u^{\prime} \right)}}}}} & (16)\end{matrix}$

[0090] where t_(k) ^(z)(u′) denotes an average texture feature valuehaving a direction k at a size z of the region u′. A weight value w^(z)denotes the weighting coefficient shown in Equation (13). Furthermore,p(u′ v′) in Equation (14) is expressed by Equation (17): $\begin{matrix}{{p\left( {u^{\prime},v^{\prime}} \right)} = {F\left\lbrack \frac{\min \left\lbrack {{P\left( u^{\prime} \right)},{P\left( v^{\prime} \right)}} \right\rbrack}{P_{O}} \right\rbrack}} & (17)\end{matrix}$

[0091] where P_(o) denotes a threshold value for the sizes of segmentedregions marked in a fundamental region division map. s(u′, v′) inEquation (14) is expressed by Equation (18): $\begin{matrix}{{s\left( {u^{\prime},v^{\prime}} \right)} = {0.5 + {0.5{F\left\lbrack \frac{\max \left\lbrack {{S\left( u^{\prime} \right)},{S\left( v^{\prime} \right)}} \right\rbrack}{S_{O}} \right\rbrack}}}} & (18)\end{matrix}$

[0092] where S_(o) denotes a saturation threshold value. Functions F inEquations (17) and (18) are used to suppress an adverse effect caused byextremely small-sized or low saturation regions. The texture weightvalue calculator 222 shown in FIG. 11 calculates a texture weight valuew_(t) (u′, v′) as expressed by Equation (19) and outputs the calculatedtexture weight value to the second multiplier 226:

w _(t)(u′, v′)=[1−s(u′, v′)](ŵ _(c) +ŵ _(t))+ŵ _(t) s(u′, v′)t(u′,v′)p(u′, v′)  (19)

[0093] The color weight value calculator 220 and the texture weightvalue calculator 222 shown in FIG. 11 input an average texture featurevalue through an input terminal IN6 (the input terminal IN6 correspondsto the input terminal IN5 shown in FIG. 10). Also, the color weightvalue calculator 220 and the texture weight value calculator 222 mayreceive as input required values among the color and texture weightingconstants, the maximum of a texture degree, the average texture, theweighting coefficient, and the threshold values for the sizes andsaturations of segmented regions marked in a fundamental region divisionmap from the outside through the input terminal IN6, or may prestorethem.

[0094] The first multiplier 224 multiplies the color weight value w_(c)(u′, v′) output from the color weight value calculator 220 by a colordistance D_(c) as defined by Equations (10), (11) or (12) and outputsthe multiplication result to the adder 228. The second multiplier 226multiplies the texture weight value w_(t) (u′, v′) output from thetexture weight value calculator 222 by a texture distance D_(t) asdefined by Equation (13) and outputs the multiplication result to theadder 228. The adder 228 adds the multiplication result from the secondmultiplier 226 to the multiplication result from the first multiplier224 and outputs the result as expressed by Equation (20) to the imagegraph simplification unit 38 through an output terminal OUT3 as acolor-texture distance {circumflex over (D)}(u′, v′) required forsimplifying an image graph.

{circumflex over (D)}(u′, v′)=w _(c)(u′, v′)D _(c)(u′, v′)+w _(t)(u′,v′)  (20)

[0095] where D_(c)(u′, v′) and D_(t)(u′, v′) denote color and texturedistances, respectively.

[0096] As described above, the color weight value w_(c) and the textureweight value w_(t), which are utilized for calculating the color-texturedistance used in the image graph simplification unit 38, are notpredetermined values but variables. This is because the color-texturedistance needs to be adjusted if the texture degrees, sizes andsaturations of segmented regions vary. For example, if the sizes p(u′,v′) of the segmented regions u′ and v′ are less than the threshold valueP_(o) for the sizes of segmented regions, the color weight value w_(c)(u′, v′) shown in Equation (20) increases while the texture weight valuew_(t) (u′, v′) decreases. If the texture degrees t(u′, v′) of thesegmented regions u′ and v′ increase, the color weight value w_(c) (u′,v′) decreases while the texture weight value w_(t) (u′, v′) increases.

[0097] A process of performing step 30 by the image graph simplificationunit 48 using the color-texture distance calculated as described abovewill now be described.

[0098] First, regions marked in the image graph output from the mainregion section unit 36 are arranged in order of decreasing size. Next,the color-texture distance between an arbitrary central region(hereinafter called “base region”) and each of adjacent regionsconnected to the base region among the arranged regions is calculatedusing Equation (20). If the calculated color-texture distance is lessthan or equal to the second threshold value th2, the base region and theadjacent regions are merged. That is, the adjacent regions are mergedinto the base region. If the base region embraces its adjacent regionsin this way, a portion of the image graph relating to the base regionsuch as a connection relationship between the base region and eachadjacent region needs to be modified as features of the base region suchas color, texture, size, and edges change. The merging of a newlymodified base region and its adjacent regions can be performed in thesame manner. When the process of modifying the base region and mergingadjacent regions into the modified base region is finished, the nextlargest region is newly designated as a base region to perform theregion merging process again. The merging process is repeated until ithas been performed on all regions marked in the image graph.

[0099] A process of simplifying the image graph cannot be perfectlyconducted regardless of the second threshold value th2. To overcome thisproblem, the image graph simplification unit 38 may be divided into aplurality of sub-region merging portions. For this purpose, the colorand texture weight values and the color and texture distances inEquation (20) are different in each sub-region merging portion, and thesecond threshold value th2 dynamically changes depending on the size ofa region.

[0100] More specifically, FIG. 12 is a block diagram of the image graphsimplification unit 38. Referring to FIG. 12, the image graphsimplification unit 38 includes primary, secondary, and tertiary regionmerging portions 240, 242 and 244,and a threshold value generator 246.The threshold value generator 246 dynamically changes the secondthreshold value th2 as expressed by Equation (21) and outputs the secondthreshold value th2 to the primary, secondary, and tertiary regionmerging portions 240, 242, and 244: $\begin{matrix}{{th2} = {\alpha^{\prime}\left( {\frac{1}{P^{\prime} + P_{O}^{\prime}} + \beta^{\prime}} \right)}} & (21)\end{matrix}$

[0101] Here, P′ denotes the size of a smaller region of two comparedregions, and P′_(o) is a constant representing a predetermined size thatthe smaller region can typically have. α denotes a threshold valueconstant and α′β′ is a threshold value for a larger region of the twocompared regions.

[0102] The primary region merging portion 240 compares the color-texturedistance as defined by Equation (20) input through an input terminal IN8with the second threshold value th2 received from the threshold valuegenerator 246, merges two segmented regions u′ and v′ marked in theimage graph input through an input terminal IN7 if the color-texturedistance is less than or equal to the second threshold value th2, andoutputs a first intermediate image graph reflecting the final result ofmerging in this way to the secondary region merging portion 242. Theperformance of the primary region merging portion 140 can be improved bygradually increasing the threshold value constant α′ in Equation (21).

[0103] The secondary region merging portion 242 compares a firstcolor-texture distance input through the input terminal IN8 with thesecond threshold value th2 received from the threshold value generator246, merges two regions u″ and v″ marked in the first intermediate imagegraph if the first color-texture distance is less than or equal to thesecond threshold value th2, and outputs a second intermediate imagegraph reflecting the final result of merging in this way to the tertiaryregion merging portion 244. Here, the first color-texture distance isobtained by substituting into Equation (20) color and texture weightvalues calculated using Equations (14) and (19), respectively, when acolor weighting constant Ŵ_(c) is set to “0”, and texture and colordistances calculated using Equation (13) and Equation (10), (11) or(12), respectively, when W_(H)<<W_(B), and W_(S)<<W_(B). That is, thefirst and second multipliers 224 and 226 of the color-texture distancegenerator 214 shown in FIG. 11 multiply a color weight value w_(c) (u″,v″) and a texture weight value w_(t) (u″, v″) output from the colorweight value calculator 220 and the texture weight value calculator 222,respectively, when the color weighting constant ŵ_(c) is ‘0’, by a colordistance and a texture distance, respectively, which are output from thecolor and texture distance calculators 210 and 212, respectively, whenW_(H)<<W_(B), and W_(S)<<W_(B). In this case, the adder 228 adds themultiplication results to each other, and outputs the addition result tothe image graph simplification unit 38 through the output terminal OUT3as the first color-texture distance. This process can be performed onlywhen the regions u″ and v″ have very low brightness.

[0104] The tertiary region merging portion 244 compares a secondcolor-texture distance input through an input terminal IN8 with thesecond threshold value th2 received from the threshold value generator246, merges two regions u′″ and v′″ marked in the second intermediateimage graph if the second color-texture distance is less than or equalto the second threshold value th2, and outputs a final image graphreflecting the final result of merging in this way through an outputterminal OUT4. Here, the second color-texture distance is obtained bysubstituting into Equation (20) color and texture weight valuescalculated when Ŵ_(c)<<Ŵ_(t) from the Equations (14) and (19),respectively, and color and texture distances calculated from Equation(10), (11) or (12) and Equation (13), respectively That is, the firstand second multipliers 224 and 226 of the color-texture distancegenerator 214 shown in FIG. 11 multiply a color weight value w_(c) (u′″,v′″) and a texture weight value ŵ_(t) (u′″, v′″) output whenŴ_(t)<<Ŵ_(c) from the color weight value calculator 220 and the textureweight value calculator 222, respectively, by a color distance and atexture distance, respectively. In this case, the adder 228 adds themultiplication results to each other, and outputs the addition result tothe image graph simplification unit 38 through the output terminal OUT3as the second color-texture distance. The tertiary region mergingportion 244 can perform this process only when the regions u′″ and v′″have large texture degrees.

[0105]FIGS. 13A and 13B show an input or preprocessed image, and regionsof the image sectioned by a method and apparatus for sectioning an imageinto a plurality of regions according to the present invention,respectively. FIGS. 14A and 14B show another input or preprocessedimage, and regions of the image sectioned by a method and apparatus forsectioning an image into a plurality of regions according to the presentinvention, respectively. FIG. 15A and 15B show yet another input orpreprocessed image, and regions of the image sectioned by a method andapparatus for sectioning an image into a plurality of regions accordingto the present invention, respectively. The image shown in FIGS. 13A,14A, or 15A is segmented into a plurality of clusters of the sameregions in the main region section unit 36, and the image graphsimplification unit 38 merges the segmented regions in order to simplifythem. In this case, a final image graph having information about thesimplified result is used to obtain the image sectioned as shown in FIG.13B, 14B, or 15B.

[0106] The method and apparatus for sectioning an image into a pluralityof regions according to the present invention can be effectively used indigital interactive video systems. In concert with the popularization ofdigital broadcasting, much research is focused on developing a digitalinteractive video system which enables a user to directly and activelyexchange information with a broadcaster instead of passively obtaininginformation by watching a broadcast. For example, in a digitalinteractive video system, if a user directly selects a desired object ona screen in TV dramas, movies, educational programs, advertisements,shopping programs, auctions, etc., which are broadcast, in order toobtain information about the desired object, the information isdisplayed on the screen. For example, the on-screen wardrobe of popularsingers, furniture and characters displayed on TV dramas, ingredientsand recipes required for cooking programs, and so on, may be meaningfulto the user. If the user selects one of these meaningful objects,information about the selected object can be provided to the user. Toaccomplish this, meaningful objects existing in broadcast images need tobe effectively distinguished. To effectively differentiate a regionoccupied by a meaningful object from a background or other objects,various features of the meaningful object should be utilized to thefullest. The features of the selected object may include color, textureand shape. In particular, color or texture gives a clue to simplifyingthe object into a single region. The method and apparatus for sectioningan image into a plurality of regions according to the present inventionincludes the use of a mean-shift analysis using all information aboutcolor feature values, texture feature values, and position on an imageplane for each pixel. Thus, this invention provides a final image graphwhich can stably and clearly distinguish regions of image including thesame color and texture feature values as well as regions including athree-dimensional object having shades gradually changing along anL-axis.

[0107] Furthermore, in a case where a selected object is composed ofvarious colors or various textures, the method and apparatus forsectioning an image into a plurality of regions according to the presentinvention makes it possible to effectively distinguish a region of themeaningful object from other regions.

[0108] In addition, the method and apparatus for sectioning an imageinto a plurality of regions according to the present invention may alsoserve as a basic module in products and software relating to digitalbroadcasting.

[0109] Meanwhile, the method and apparatus for sectioning an image intoa plurality of regions according to the present invention caneffectively be used in searching for and recognizing a meaningful objectexisting in an image and tracking the meaningful object. That is,information about a region occupied by the meaningful object isextracted using a final image graph generated by the method andapparatus according to this invention. In this case, this invention canbe used in searching or recognizing a screen on which an object havinginformation similar to the extracted information was previouslydisplayed, and tracking all screens on which an object havinginformation similar to the extracted information was previouslydisplayed. Furthermore, the extracted information may also be used inediting or manufacturing an image.

[0110] Furthermore, if only meaningful objects in an image to becompressed according to image compression standards such as MPEG are tobe effectively transmitted, this invention can maximize transmissionefficiency by compressing only extracted information more finely.

[0111] As described above, the method and apparatus for sectioning animage into a plurality of regions makes it possible to stably extractregions of an image having various features, that is, an image havingshades, an image including a texture, or image of various colordistributions in a form similar to what humans can perceive.Furthermore, this invention has a wide variety of applications asdescribed above.

What is claimed is:
 1. A method for sectioning an input image into aplurality of regions, the method comprising the steps of: (a) convertingeach pixel of the input image into color coordinates in an arbitrarycolor space including a luminance (L) axis; (b) partitioning the colorspace into a plurality of layers along the luminance (L) axis andpartitioning each of the plurality of layers into a plurality of binsalong color (α and β) axes; (c) obtaining an upper last circle by amean-shift analysis, the upper last circle encircling the largest numberof pixels having features most similar to those of a base pixel locatedat a predetermined position within a bin having the largest number ofpixels in the color space among bins included in an upper layer; (d)obtaining a lower last circle by the mean-shift analysis, the lower lastcircle including the largest number of pixels having features mostsimilar to those of a base pixel located at the same position as thecenter of the upper last circle in a lower layer adjacent to the upperlayer; (e) including the upper last and lower last circles in the samecluster if a distance between the centers of the upper last and lowerlast circles is less than a first threshold value, and if another lowerlayer adjacent to the lower layer exists, determining the lower layerand the adjacent lower layer as a new upper layer and a new lower layer,respectively, and returning to step (d); (f) if the distance between thecenters of the upper last and lower last circles is no less than thefirst threshold value or if the adjacent lower layer does not exist,partitioning the cluster using a last circle including the smallestnumber of pixels in the color space used in calculating an averagecolor, an average texture, and an average position among the lastcircles included in the cluster; and (g) performing steps (c)-(f) on theremaining pixels not included in the upper last or the lower last circleand generating an image graph using clusters obtained for all pixels inthe color space.
 2. The method of claim 1, further comprising the stepsof smoothing the input image a predetermined number times before step(a), wherein an arbitrary color space including the luminance (L) axisis an Lαβ color space, and in step (a), each pixel in the smoothed imageis converted into color coordinates in the Lαβ color space.
 3. Themethod of claim 2, after step (g), further comprising the step ofcomparing a color-texture distance with a second threshold value andmerging regions marked in the image graph according to the comparisonresult to obtain a final image graph.
 4. The method of claim 3, whereinstep (c) comprises the steps of: (c1) among pixels included in an upperarbitrary circle centered on the predetermined position, selecting onlypixels whose texture feature values are less than a texture featurevalue of a pixel located at the center of the upper arbitrary circle andwhich are spaced apart less than a predetermined distance from theposition of the pixel located at the center of the upper arbitrarycircle on an image plane, and obtaining the average color, the averagetexture, and the average position on the image plane for the selectedpixels; (c2) determining whether a spacing degree obtained fromdifferences between the averages and feature values of the base pixel atthe center of the upper arbitrary circle is less than or equal to afirst predetermined value in the Lαβ color space; (c3) designating apixel located at the position corresponding to the calculated averagecolor as an imaginary base pixel if the spacing degree is greater thanthe first predetermined value, designating a texture feature value and aposition on the image plane of the imaginary base pixel as the averagetexture and the average position on the image plane calculated in step(c1), determining a new upper arbitrary circle centered on thedesignated imaginary base pixel, and returning to step (c1); and (c4)determining the upper arbitrary circle as the upper last circle if thespacing degree is less than or equal to the first predetermined valueand proceeding with step (d).
 5. The method of claim 4, wherein step (d)comprises steps of: (d1) among pixels included in a lower arbitrarycircle having its center at the same position as the center of the upperlast circle in the lower layer, selecting only pixels whose texturefeature values are less than a texture feature value of a base pixellocated at the same position as the center of the upper last circle andwhich are spaced apart less than a predetermined distance from aposition of the base pixel on the image plane, and obtaining an averagecolor, an average texture, and an average position on the image planefor the selected pixels; (d2) determining whether a spacing degreeobtained from differences between the averages calculated in the step(d1) and feature values of the base pixel at the center of the lowerarbitrary circle is less than or equal to the first predetermined valuein the Lαβ color space; (d3) designating a pixel located at a positioncorresponding to the average color calculated in the step (d1) as animaginary base pixel if the spacing degree of step (d2) is greater thanthe first predetermined value, designating the texture feature value anda position on the image plane of the imaginary base pixel as the averagetexture and the average position on the image plane calculated in thestep (d1), determining a new lower arbitrary circle centered on thedesignated imaginary base pixel, and returning to the step (d1); and(d4) determining the lower arbitrary circle as the lower last circle ifthe spacing degree of step (d2) is less than or equal to the firstpredetermined value and proceeding with step (e).
 6. The method of claim4, wherein, in step (c1), if a distance Δ′ between any pixel located inthe upper arbitrary circle and a pixel located at the center of theupper arbitrary circle is less than a second predetermined value, anaverage color, a average texture, and an average position on an imageplane for any pixel in the upper arbitrary circle are reflected whencalculating the averages, and wherein the distance Δ′ between the pixelsis calculated according to the following equation:$\Delta^{\prime} = {\frac{{\Delta \quad X^{2}} + {\Delta \quad Y^{2}}}{\sigma_{G}^{2}} + \frac{{\Delta\alpha}^{2} + {\Delta\beta}^{2}}{\sigma_{C}^{2}} + {\sum\limits_{i^{\prime} = 1}^{K}\frac{\sum\limits_{j^{\prime} = 1}^{Z}{\Delta\theta}_{i^{\prime}j^{\prime}}^{2}}{\sigma_{\theta_{i^{\prime}}}^{2}}}}$

 where X and Y are positions on the image plane for each of the pixels,α and β are color components in the color space, θ denotes a textureresponse, and σ_(G), σ_(C) and σ_(θ) are the predetermined distance bywhich the pixels are spaced apart from the position of the base pixel, aradius of the upper arbitrary circle, and a texture distancerepresenting the extent to which the texture feature value of the pixelis less than that of the base pixel, respectively.
 7. The method ofclaim 5, wherein, in step (d1), if a distance Δ′ between any pixellocated in the lower arbitrary circle and a pixel located at the centerof the lower arbitrary circle is less than a second predetermined value,an average color, an average texture, and an average position on animage plane for any pixel in the lower arbitrary circle are reflected incalculating the averages, and wherein the distance Δ′ between the pixelsis calculated according to the following equation:$\Delta^{\prime} = {\frac{{\Delta \quad X^{2}} + {\Delta \quad Y^{2}}}{\sigma_{G}^{2}} + \frac{{\Delta\alpha}^{2} + {\Delta\beta}^{2}}{\sigma_{C}^{2}} + {\sum\limits_{i^{\prime} = 1}^{K}\frac{\sum\limits_{j^{\prime} = 1}^{Z}{\Delta\theta}_{i^{\prime}j^{\prime}}^{2}}{\sigma_{\theta_{i^{\prime}}}^{2}}}}$

 where X and Y are positions on the image plane for each of the pixels,α and β are color components in the color space, θ denotes a textureresponse, and σ_(G), σ_(C) and σ_(θ) are the predetermined distance bywhich the pixels are spaced apart from the position of the base pixel onthe image plane, a radius of the lower arbitrary circle, and a texturedistance representing the extent to which the texture feature value ofthe pixel is less than that of the base pixel, respectively.
 8. Themethod of claim 1, wherein the upper layer has a brightness level higherthan a brightness level of the lower layer.
 9. The method of claim 1,wherein the upper layer has a brightness level lower than a brightnesslevel of the lower layer.
 10. An apparatus for sectioning the inputimage into the plurality of regions, comprising: an image preprocessorthat smooths the input image a predetermined number of times, enhancesedges in the smoothed image, and outputs a preprocessed image; a featurevalue calculator that calculates color feature values and texturefeature values from the image output from the image preprocessor on apixel-by-pixel basis and outputs the calculated result; a main regionsection unit that generates an image graph from clusters obtained byusing color, texture, and position on the image plane for each pixel inthe image output from the image preprocessor and the color and texturefeature values output from the feature value calculator and outputs thegenerated image graph; and an image graph simplification unit thatcompares a color-texture distance calculated using the color featurevalues and the texture feature values output from the feature valuecalculator with a second threshold value, simplifies the image graphaccording to the comparison result, and outputs a final image graphobtained from the simplified image graph.